Image compression apparatus

ABSTRACT

To improve the data compression ratio in the data compression/decompression method utilizing the down-sampling process. Image data in a DS area division unit ( 130 ) is divided into a plenty of DS areas and DS areas which are not important are converted into contracted data of the block size by down-sampling in a down-sampling unit ( 131 ). A relocation unit ( 132 ) allocates two or more contracted data in one DS area, inserts file data into the other portion in the DS area, and removes the DS area where no contracted data is allocated, thereby reducing the entire image size. The image data thus contracted is inputted into a JPEG encoder ( 14 ) and subjected to a DCT process and a quantization process for each block. When performing restoration, decompression is performed by the JPEG decoder and the original location is restored. As for the contracted data, interpolation is performed to restore the image data.

This application is a 371 of PCT/JP05/02259 filed Feb. 15, 2005.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to an image compressing and/or expandingmethod, an image compression apparatus, and an image expansionapparatus. More specifically, the present invention relates to animprovement of an image compressing and/or expanding method such as JPEGcompressing and/or expanding method for performing orthogonal transformand quantization on image data for every block region obtained bydividing the image data into a plurality of block regions.

BACKGROUND ART

As methods for compressing and/or expanding image data constituted by astill image, there are widely known JPEG (Joint Photographic ExpertGroup) standards standardized according to CCITT (InternationalTelegraph and Telephone Consultative Committee) and ISO (InternationalStandards Organization). In the JPEG standards, an image datacompressing method performed by dividing a frame image into a pluralityof blocks in such a way that 8×8 pixels constitute one block, andtransforming a spatial coordinate into a frequency coordinate, and animage data expanding method thereof are defined.

A data compressor according to the JPEG standards (hereinafter call“JPEG compressor”) divides input image data into many blocks, andperforms DCT (Discrete Cosine Transform) processing and quantizationprocessing on each block. In this quantization processing, a valueobtained by multiplying data specified for each DCT coefficient by aquantization table by a quantization factor Q is used as a quantizationstep width. The DCT coefficient obtained by the DCT processing isquantized by the quantization step width, thereby irreversibly reducinga data amount. Thereafter, entropy coding using a run-length processing,a differential processing, a Huffman coding processing or the like isperformed, thereby generating compressed image data. This coding is aprocessing for irreversibly reducing the data amount.

On the other hand, a data expander according to the JPEG standards(hereinafter call “JPEG expander”) performs opposite processings tothose performed by the JPEG compressor to restore compressed image datato original image data. Namely, input compressed image data is decodedand dequantized using the same quantization table and the samequantization factor Q as those used in the data compression. Thereafter,an inverse DCT processing unit performs an inverse DCT transform tocombine the divided blocks, thereby restoring the compressed image datato the original image data.

To improve a data compression rate of the JPEG compressor, it isnecessary to change the quantization table or the quantization factor Qso as to make the quantization step width larger. However, if a largedata amount is reduced in the quantization processing, which is anirreversible processing, a quality of the restored image data is greatlydegraded. In addition, this quality degradation occurs throughout theimage. Due to this, even if an important region and an unimportantregion are present in the image, the image quality is disadvantageously,uniformly degraded in the both regions.

To solve this problem, there has been proposed a compression processingmethod for making a quality of an image after restoration differentamong regions of the image (see, for example, Patent Document 1). ThePatent Document 1 discloses a data compression processor that includes amask circuit that masks a DCT coefficient before a quantizationprocessing. This data compression processor makes a mask employed in themask circuit different among the regions, thereby coding an image in animportant region at a high image quality and coding the image in anunimportant region at a low image quality.

However, this data compression processor needs to perform a maskprocessing halfway along DCT processing and quantization processingthose are performed sequentially. Due to this, a general-purpose JPEGcompression processor such as a JPEG chipset can not be employed as thisdata compression processor and the data compression processor isdisadvantageously made expensive. Furthermore, the mask processing is aprocessing performed in each block, so that there is a limit to a dataamount that can be reduced.

Patent Document 1: Japanese Unexamined Patent Publication No.1994-054310

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention

To solve the above-stated problems, the inventors of the presentapplication filed a prior patent application (identified by JapanesePatent Application No. 2003-43367). This prior application discloses atechnique for dividing image data into a plurality of small regions andfor performing downsampling processing on a part of the small regions asa preprocessing to JPEG compression processing. This downsampling is aprocessing for reducing image data in the divided small region bydownsampling, making the reduced image data coincident with the block,and inserting fill data into a remaining part of the region. By usingthis method, it is possible to reduce the data amount while making thequality of the image different among the regions by using thegeneral-purpose JPEG compression processor.

The present invention has been achieved in view of these circumstances.It is an object of the present invention to improve a data compressionrate of an image compression apparatus using the downsamplingprocessing. It is also an object of the present invention to improve adata compression rate of an image compression apparatus that can use ageneral-purpose data compressor and a general-purpose data expander andthat reduces a data amount while making a quality of an image differentamong regions. It is particularly an object of the present invention torealize a high compression rate using a JPEG compressor and a JPEGexpander.

Means for Solving the Problems

An image compression apparatus according to a first aspect of thepresent invention includes downsampling means, rearranging means, anddata compressing means. The downsampling means divides image data into aplurality of downsampling regions each at a size that is an integermultiple of a block size. The downsampling means performs downsamplingon at least a part of said downsampling regions to decrease the numberof pixels thereof, and generates reduced data each at a size that is aninteger multiple of said block size. The rearranging means arranges twoor more pieces of said reduced data in each of downsampling regionssubjected to said downsampling, inserts fill data into each of pixelsleft in said downsampling region in which said reduced data is arranged,and removes the downsampling regions in which the reduced data is notarranged. The data compressing means divides image data generated by therearranging means into a plurality of block regions each at said blocksize, and performs a compression processing including an orthogonaltransform and a quantization on each of said block regions, therebygenerating compressed image data. With this constitution, downsamplingand rearrangement may are performed for a part of the image region,thereby the image data can be spatially compressed as a preprocessing toa data compressing step.

Further, said downsampling region dividing means divides said image datainto said downsampling regions having different sizes, and saiddownsampling means performs said downsampling at different reductionrates according to the sizes of said downsampling regions, therebygenerates said reduced data equal in size. With this constitution, acombination of downsamplings at different reduction rates can be used,and a data compression rate can be improved.

In an image compression apparatus according to a second aspect of thepresent invention, the downsampling means performs downsampling on atleast a part of said downsampling regions to generate said reduced data,collects said reduced data to generate a new downsampling region,performs the downsampling again on the new downsampling region, therebygenerates new reduced data each at a size that is an integer multiple ofsaid block size. With this constitution, a combination of downsamplingsat different reduction rates can be used.

An image compression apparatus according to a third aspect of thepresent invention performs a compression processing on image data thatconsists of luminance data and color-difference data. The imagecompression apparatus includes downsampling region dividing means,luminance downsampling means, luminance data rearranging means,color-difference downsampling means, color-difference data rearrangingmeans, and data compressing means. The downsampling region dividingmeans divides said luminance data that constitutes the image data intoluminance downsampling regions each at a size that is an integermultiple of a block size, and divides the color-difference data thatconstitutes the image data into color-difference downsampling regionseach at a size that is the integer multiple of said block size.

The luminance data divided into the luminance downsampling regions isprocessed by the luminance downsampling means and the luminance datarearranging means. The luminance downsampling means performsdownsampling on at least a part of said luminance downsampling regionsto decrease the number of of said luminance data, and generates reducedluminance data each at a size that is an integer multiple of said blocksize. The luminance data rearranging means arranges two or more piecesof said reduced luminance data in each of luminance downsampling regionssubjected to said luminance downsampling, and inserts fill data intoeach of pixels left in said luminance downsampling region in which saidreduced luminance data is arranged. During rearranging, the luminancedata rearranging means arranges the reduced luminance data in theluminance downsampling regions so as to leave a discrimination regionhaving said block size at a predetermined position in each of saidluminance downsampling regions.

On the other hand, the color-difference data divided into thecolor-difference downsampling regions is processed by thecolor-difference sampling means and the color-difference datarearranging means. The color-difference downsampling means performsdownsampling on said color-difference downsampling regions determinedbased on said luminance downsampling regions subjected to said luminancedownsampling to decrease the number of said color-difference data, andgenerates reduced color-difference data each at a size that is theinteger multiple of said block size. The color-difference datarearranging means arranges two or more pieces of said reducedcolor-difference data in each of said color-difference downsamplingregions.

With this constitution, it is unnecessary to put the discriminationregion in each of the color-difference downsampling regions in which thereduced color-difference data is rearranged, therefore, it is possibleto improve compression efficiency for compressing the image data thatconsists of the luminance data and the color-difference data.

An image compression apparatus according to a fourth aspect of thepresent invention, in addition to the third aspect of the presentinvention, further includes color-difference data dividing means fordividing each of said color-difference downsampling regions that are notsubjected to said color-difference downsampling into regions each at asize equal to said reduced color-difference data, thereby generatingdivided color-difference data. In addition, said color-difference datarearranging means is constituted to rearrange said reducedcolor-difference data and the divided color-difference data.

With this constitution, not only the reduced data but also the unreduceddata can be rearranged with respect to the color-difference data,therefore, it is possible to further improve the compression efficiencyfor compressing the image data that consists of the luminance data andthe color-difference data.

Effect of the Invention

According to the present invention, it is possible to improve the datacompression rate of the image compression apparatus using thedownsampling processing. In addition, it is possible to improve the datacompression rate of the image compression apparatus that can use ageneral-purpose data compressor and a general-purpose data expander andthat reduces the data amount while making a quality of an imagedifferent among regions. It is particularly possible to realize a highcompression rate using the JPEG compressor and the JPEG expander.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

<Image Transmission System>

FIG. 1 is a block diagram of an example of a configuration of an imagecompression and expansion system according to the first embodiment ofthe present invention and shows an example of an image transmissionsystem. This image transmission system includes a transmission side unitUt and a reception side unit Ur connected to each other by acommunication network 100 and can compress image data and transmit thecompressed image data from the transmission side unit Ut to thereception side unit Ur. In this embodiment, image data of an image inputapparatus 101 connected to the transmission side unit Ut is transmittedto an image output apparatus 102 connected to the reception side unitUr.

The communication network 100, which is constituted by a wired orwireless communication line for transmitting digital data, includes anexchange unit and/or a repeater unit if it is necessary to do so. Forexample, a packet communication network such as Ethernet®, the Internetor ATM (Asynchronous Transfer Mode) network, or the other digitalnetwork can be used as the communication network 100.

The image input apparatus 101, which is an apparatus that provides imagedata, includes, for example, an imaging device such as a camera, animage reader such as a scanner, and a data storage device such as a HDD(Hard Disc Drive). In the present embodiment, the image input apparatus101 is assumed to generate image data of a still image in an RGB formatand to output the generated image data to the transmission side unit Ut.In the present specification, the still image means an image constitutedby many pixels arranged two-dimensionally, and examples of the stillimage include a frame image of each frame and a differential imagebetween frames that constitutes a moving image.

The image output apparatus 102, which is an apparatus that uses theimage data output from the reception side unit Ur, includes, forexample, a display device such as an LCD, an image forming device suchas a printer, and a data storage device such as the HDD (Hard DiscDrive).

<Transmission Side Unit>

FIG. 2 is a block diagram of an example of a configuration of thetransmission side unit Ut shown in FIG. 1. This transmission side unitUt includes a YUV converter 10, an image compressor 11, and a datatransmitter 12. The image data generated by the image input apparatus101 is converted first into image data in a YUV format in the YUVconverter 10. The resultant image data in the YUV format is compressedby the image compressor 11 into compressed data having a reduced dataamount. This compressed data is transmitted to the communication network100 by the data transmitter 12.

The YUV converter 10 is format converting means for converting the imagedata in the RGB format into the image data in the predetermined YUVformat (e.g., YUV410, 411, 420, 422 or 444). If the image data in theYUV format is input from the image input apparatus 101 or JPEGcompression is performed on the image data in the RGB format, the YUVconverter 10 may be omitted.

The image compression unit 11 includes the image reduction converter 13and the JPEG encoder 14. The image reduction converter 13 divides aninput image into many small regions, reduces a resolution at least forpart of the small regions, and rearranges these small regions, therebyreducing a size of the entire image. This resolution reduction isperformed for small region by small region according to a image qualityrequired for each region. The JPEG encoder 14 subjects the image data toa compression processing according to the JPEG standards (“JPEGcompression processing”), and generates the compressed data having afurther reduced data amount.

<Reception Side Unit>

FIG. 3 is a block diagram of an example of a configuration of thereception side unit Ur shown in FIG. 1. This reception side unit Urincludes an RGB converter 22, an image expander 21, and a data receiver20. The compressed data transmitted from the transmission side unit Utto the communication network 100 is received by the data receiver 20.The received compressed data is expanded by the image expander 21 andrestored to the image data in the YUV format. This image data isconverted into image data in the RGB format by the RGB converter 22 andthe image data in the RGB format is output to the image output apparatus102.

The image expander 21 includes a JPEG decoder 23 and an imageenlargement converter 24. The compressed data from the data receiver 20is subjected to an expansion processing according to the JPEG standards(“JPEG expansion processing”) by the JPEG decoder 23, and restored tothe image data before the JPEG compression. An image size of the imagedata thus expanded is increased by the rearrangement and the resolutionimprovement by the image enlargement converter 24, whereby the imagedata is restored to image data before the resolution is reduced.

The RGB converter 22 converts the image data in the YUV format outputfrom the image enlargement converter 24 into the image data in the RGBformat, and outputs the image data in the RGB format to the image outputapparatus 102. If the image data in the YUV format is input to the imageoutput apparatus 102 or the image data in the RGB format is subjected tothe JPEG compression, the RGB converter 22 is omitted.

<Image Compressor>

FIG. 4 is a block diagram of an example of a configuration of the imagecompressor 11 shown in FIG. 2. FIG. 4 shows an example of detailedconfigurations of the image reduction converter 13 and the JPEG encoder14. The image reduction converter 13 includes a downsampling regiondivider (“DS region divider”) 130, a downsampling unit 131, and arearrangement unit 132.

[DS Region Divider]

The DS region divider 130 divides the image data in the YUV formatconverted from the RGB format into a plurality of DS regions. Each DSregion includes an arbitrary number of pixels and can be set as a regionin an arbitrary shape. However, considering compression efficiency, itis preferable that a block used in the JPEG encoder 14 is set as a basicunit and that each DS region is a group of such blocks.

Since a size of a JPEG block is 8×8 pixels, a size of a DS region, whichis an integer multiple of the JPEG block, may be 16×16 pixels, 32×32pixels, 24×16 pixels, 8×32 pixels or the like. However, if a block sizediscrimination region, to be described later, is inserted into each DSregion, it is necessary to set the DS region size three times or more aslarge in area as the block size so as to reduce the entire image regionsby the rearrangement.

Furthermore, the DS regions may be different in size or different inshape. However, in views of facilitating a rearrangement processing, tobe described later, the DS regions are preferably equal in size and inshape and more preferably, the DS regions are rectangular (particularlysquare). In the present embodiment, an instance of dividing the entireimage into a plurality of DS regions equal in size, i.e., each having asize of 16×16 pixels will be described.

[Downsampling Unit]

The downsampling unit 131 performs downsampling on each pixel data thatconstitutes each DS region for at least one part of the DS regions, andconverts the image data into reduced data having reduced number ofpixels. It is preferable that a size of this reduced data is an integermultiple of the block size in view of the compression rate of the JPEGencoder, and in this embodiment, the reduced data size is madecoincident with the block size (8×8 pixels).

The DS regions subjected to the downsampling processing are determinedaccording to the image quality required for each DS region. Forinstance, if the entire image regions include important regions requiredto have a high image quality and unimportant regions that are notrequired to have the high image quality, the downsampling processing isperformed only on the unimportant DS regions. For instance, for theimportant DS region, the input data (unreduced data) is output withoutprocessing it. For the unimportant DS region, the reduced data obtainedby the downsampling processing is output.

Discrimination as to whether the region is the important region or theunimportant region, that is, discrimination as to whether the DS regionis a DS region subjected to the downsampling is determined in advance ordesignated by an operator. Alternatively, whether the region is theimportant region or the unimportant region may be automaticallydiscriminated based on an output signal of a sensor (not shown) or basedon the image data. For instance, a region including a motion can bediscriminated as the important region based on a result of a comparisonamong the frames of the moving image. In addition, a flat image regionhaving relatively small change can be discriminated as the unimportantregion.

The downsampling processing is realized by a subsampling processing forthinning pixels in each DS region, a filter processing using a low-passfilter and the like. The downsampling processing may be realized only bythe subsampling processing. However, a highly accurate downsamplingprocessing with fewer distortions can be realized by a combination ofthe filter processing and the subsampling processing. For instance, in aprocessing using a Gaussian filter as the low-pass filter, aone-dimensional Gaussian filter is applied to a horizontal direction ofthe image data and then the same filter is applied to a verticaldirection of obtained data. The image data subjected to the filterprocessing by the Gaussian filter is subjected to the subsamplingprocessing, whereby highly accurate downsampling can be performed.

FIGS. 5( a) and 5(b) show examples of a DS region division and adownsampling performed by the image compressor 11. FIG. 5( a) shows aninstance in which the entire image is divided into DS regions A1 eachhaving a size of 16×16 pixels. In FIG. 5( a), symbol BL denotes a JPEGblock having a size of 8×8 pixels. The DS region divider 130 coincides areference position for the division (upper left position of a screen inFIG. 5( a)) with that for JPEG block division so that the DS region A1is a group of the blocks BL. The downsampling unit 131 performsdownsampling on the DS region A1, which is the unimportant region, sothat the number of pixels of the DS region A1 is reduce to ½ bothvertically and horizontally and converted into reduced data having asize of 8×8 pixels. Likewise, FIG. 5( b) shows an instance of dividingthe entire image into DS regions A2 each having a size of 32×32 pixels.Each DS region A2, which is the unimportant region and which is dividedby the DS region divider 130, is converted into reduced data having asize of 8×8 pixels by downsampling the DS region A2 to reduce the numberof pixels of the DS region A2 to ¼ both vertically and horizontally bythe downsampling unit 131.

FIG. 6 is a block diagram of an example of a detailed configuration ofthe downsampling unit 131. This downsampling unit 131 is an adaptivedownsampling unit that performs downsampling based on each pixel data inthe DS region, and reduces a resolution of each DS region constituted bya flat image while maintaining a high quality of each DS regionincluding a profile or a fine change, thereby reducing the data amount.

This adaptive downsampling unit includes a low-pass filter 30, asubsampling unit 31, an output selector 32, a subtracter 33, and adownsampling determination unit (“DS determination unit”) 34. Pixel dataf₁₆(x, y) at a size of 16×16 pixels that constitutes each DS region isinput to the adaptive downsampling unit. The pixel data f₁₆(x, y) iseither luminance data Y or color-difference data U and V in a coordinate(x, y) of each DS region.

The low-pass filter 30 removes a high frequency component of the inputpixel data f₁₆(x, y), thereby providing pixel data f_(L)(x, y)consisting of a low frequency component. This pixel data f_(L)(x, y) isfurther subjected to subsampling for every other pixel by thesubsampling unit 31, thereby providing pixel data f₈(x, y) having thesize of 8×8 pixels and represented by the following equation.f ₈(x,y)=f _(L)(2x,2y) for x,y=0,1,2, . . . , 7  (1)

Namely, the low-pass filter 30 and the subsampling unit 31 perform thedownsampling processing to generate the reduced data. The outputselector 32 selects one of the pixel data f₁₆(x, y), which is unreduceddata, and the pixel data f₈(x, y), which is reduced data, and outputsthe selected pixel data to the rearrangement unit 132 as output pixeldata f_(o)(x, y).

The DS determination unit 34 determines whether to perform thedownsampling based on the high frequency component contained in each DSregion, and switching-controls the output selector 32 based on thisdetermination result. This DS determination unit 34 includes an MSH(Means Square High Frequency Component) calculator 35 and a thresholdcomparator 36. Pixel data f_(H)(x, y) consisting of the high frequencycomponent and obtained by the subtracter 33 is input to the DSdetermination unit 34. The MSH calculator 35 calculates a square sum(MSH) of the pixel data f_(H)(x, y) according to the following equation.

$\begin{matrix}{{MSH} = {\frac{1}{16 \times 16}{\sum\limits_{x = 0}^{15}{\sum\limits_{y = 0}^{15}\{ {f_{H}( {x,y} )} \}^{2}}}}} & (2) \\{{f_{H}( {x,y} )} = {{f_{16}( {x,y} )} - {f_{L}( {x,y} )}}} & (3)\end{matrix}$

The threshold comparator 36 compares the MSH obtained as represented bythe above equation with a predetermined determination threshold Th. Ifthe MSH is lower than the determination threshold Th, the thresholdcomparator 36 outputs the downsampled pixel data f₈(x, y) to the outputselector 32, and if the MSH is equal to or higher than the determinationthreshold Th, the threshold comparator 36 outputs the input pixel dataf₁₆(x, y) to the output selector 32 without processing it.

$\begin{matrix}{{f_{o}( {x,y} )} = \{ \begin{matrix}{f_{8}( {x,y} )} & {{{if}\mspace{14mu}{MSH}} < {Th}} \\{f_{16}( {x,y} )} & {otherwise}\end{matrix} } & (4)\end{matrix}$[Rearrangement Unit]

The rearrangement unit 132 rearranges the output data of thedownsampling unit 131, generates the image data on the image the entiresize of which is reduced, and outputs the image data to the JPEG encoder14. The data output from the downsampling unit 131 includes a mixture ofunreduced data each at the size of 16×16 pixels and reduced data each atthe size of 8×8 pixels. Due to this, if two or more pieces of reduceddata are collected and rearranged in the same DS region, then the numberof DS regions can be decreased, and the size of the entire image can bethereby reduced.

If the downsampling unit 131 performs the downsampling on continuous DSregions and continuously outputs reduced data, the rearrangement unit132 sequentially arranges these pieces of reduced data in the same DSregion. If pieces of reduced data are not continuous or the number ofcontinuous pieces of reduced data is small, then the rearrangement unit132 arranges the reduced data in the same DS region and then insertsfill data into a remaining part of the DS region. The fill data isarbitrary data at a size of a pixel. A region into which the fill datahas been inserted will be referred to as “fill region”.

In the DS region in which the pieces of reduced data are arranged, thefill region consisting of 8×8 pixels is always provided besides abovementioned the blank region. This fill region is a discrimination regionthat enables discrimination as to whether the DS region is a region inwhich pieces of the reduced data have been arranged, and a position ofthis fill region is set in the DS region in advance. Since anarrangement order of the reduced data in the DS region is set in advanceand each blank region is set as the fill region, the reception side unitUr can restore arrangement of the reduced data to that of the originaldata based on the fill data. Therefore, it is unnecessary to separatelytransmit information on the arrangement of the reduced data from thetransmission side unit Ut to the reception side unit Ur.

Furthermore, providing the blank region and the discrimination region asthe fill regions into which the same data has been inserted, thecompression rate at which the JPEG encoder 14 performs a compressionprocessing can be improved.

It is noted that the JPEG encoder 14 can not process image data of ashape other than the rectangular shape. Due to this, the rearrangementunit 132 adds one or more DS regions consisting only of fill data to anend of the image data in which DS regions are eliminated, therebyreshaping the image data into rectangular image data.

FIGS. 7( a) to 7(c) show an example of an operation performed by theimage reduction converter 13. It is assumed that each of the processingsperformed by the DS region divider 130, the downsampling unit 131, andthe rearrangement unit 132 is started at the upper left position of theimage data and sequentially moved from left to right in a horizontaldirection. It is also assumed that if the processing is completed up toa right end of the image data, then a processing target is shifted in adownward direction, and the same processing is repeatedly performed on aregion located right under the processed region.

FIG. 7( a) shows a DS region dividing state. The DS region divider 130divides the image data input from the image input apparatus 101 andhaving a size of 96×96 pixels into 36 DS regions (#1 to #36) each at asize of 16×16 pixels, and sequentially outputs image data of therespective DS regions to the downsampling unit 131. Among this, DSregions #22, #28, and #34 are assumed as the important regions and theother DS regions are assumed as the unimportant regions.

FIG. 7( b) shows a downsampling state. The downsampling unit 131determines whether each of the DS regions sequentially output from theDS region divider 130 is the important region or the unimportant region.If determining that a certain DS region is the important region, thedownsampling is not performed and pixel data at the size of 16×16 pixelis output. If determining that a certain DS region is the unimportantregion, the downsampling unit 131 performs downsampling on the DS regionto reduce the number of pixels to ½ both vertically and horizontally,and outputs reduced data at a size of 8×8 pixels.

FIG. 7( c) shows a data rearranging state. If the downsampling unit 131outputs the reduced data to the data rearrangement unit 132, the datarearrangement unit 132 arranges the reduced data in the DS region in anorder of upper left, upper right, and lower left. In addition, therearrangement unit 132 inserts fill data into 8×8 pixels at the lowerright position of the DS region as the fill region (discriminationregion). In FIG. 7( c), the reduced DS regions #1 to #3 are arranged ina first DS region and the reduced DS regions #4 to #6 are arranged in anext DS region. Thereafter, the continuous pieces of reduced data #1 to#21 are similarly arranged in the respective DS regions three by three.

If the downsampling unit 131 outputs the unreduced data #22 to therearrangement unit 132, the data rearrangement unit 132 arranges thedata #22 in a new DS region. Thereafter, continuous pieces of reduceddata #23 to #27 are output again to the rearrangement unit 132, andarranged in the respective DS regions three by three. At the moment thedownsampling unit 131 outputs the unreduced data #28, only two pieces ofreduced data (#26 and 27) are arranged in a previous DS region. Due tothis, the rearrangement unit 132 inserts the fill data into a lower leftblank region in the DS region as the fill region.

Next, if the downsampling unit 131 sequentially outputs the pieces ofreduced data #29 to #33 and the unreduced data #34 to the rearrangementunit 132, the rearrangement unit 132 similarly arranges these pieces ofdata. When the last reduced data #36 is arranged in the DS region, ablank region (lower left region) remains in this DS region, therefore,the fill data is inserted into this blank region. Finally, therearrangement unit 132 adds three DS regions each consisting of the filldata only to the end of the resultant image data, and reshapes theentire image data into rectangular image data, thus finishing thisprocessing.

In reducing the image data, the rearrangement unit 132 coincides thehorizontal size of the reduced image data (the number of pixels in thehorizontal direction of the reduced image data) with the size of theoriginal image data, and reduces only the vertical size of the reducedimage data. In this way, if the size of the image data in eachprocessing direction of the rearrangement unit 132 is determined basedon the original image data, the image enlargement converter 24 canaccurately restore the compressed image data to the original image datawithout information on the size of the original image data. It is,therefore, unnecessary to separately transmit the information on theimage size from the transmission side unit Ut to the reception side unitUr.

Furthermore, the image data in the YUV format consists of one luminancedata Y and two pieces of color-difference data U and V. Due to this, theDS region divider 130, the downsampling unit 131, and the rearrangementunit 132 perform their respective processings stated above on each ofthese three pieces of data.

The data output from the downsampling unit 131 to the rearrangement unit132 may be only reduced data or unreduced data or may be data havingfill data inserted into each blank region of each downsampled DS region(that is, data equal in region size to the original image data).

<JPEG Encoder>

The JPEG encoder 14 includes a block divider 140, a DCT processor 141, aquantizer 142, a coding unit 143, a quantization table T1, and a codingtable T2 (see FIG. 4). The image data output from the image compressionconverter is divided into a plurality of blocks each at a size of 8×8pixels by the block divider 140. The DCT processor 141 performs adiscrete cosine transform (DCT) on each divided block and obtains a DCTcoefficient for each block. The respective DCT coefficients thusobtained are quantized by the quantizer 142 using the quantization tableT1.

FIG. 8 shows examples of the quantization table T1. Data for specifyingquantization step widths is shown in a matrix for every frequencycomponent in the horizontal direction and the vertical direction.Normally, different quantization tables are used between thequantization processing on the luminance data and that on thecolor-difference data. FIG. 8( a) shows an example of the quantizationtable for the luminance data whereas FIG. 8( b) shows an example of thequantization table for the color-difference data. Each of these datatables includes data for making the quantization step width larger ifthe frequency is higher. This tendency is more conspicuous in thequantization table for the color-difference data than that for theluminance data.

At the time of quantizing each DCT coefficient, the quantizationprocessor 142 reads data according to the DCT coefficient from thequantization table T1, multiplies the read data by the quantizationfactor (quantization coefficient) Q, and uses this multiplication resultas the quantization step width for the DCT coefficient. Thisquantization factor Q is an arbitrary value for adjusting thecompression rate and the image quality and given in advance. If thequantization factor Q is made higher, then the quantization step widthis increased, and the data compression rate can be improved. However,this is accompanied by generation of a block distortion and degradationof the image quality.

If the adaptive downsampling unit 131 shown in FIG. 6 is used, the dataamount of the JPEG compressed data can be reduced by setting thedetermination threshold Th used in the threshold comparator 36 higher.It is, therefore, possible to obtain a desired compression rate using arelatively low quantization factor Q. However, if this determinationthreshold Th is made excessively high, a deterioration occurs even tothe region including the profile or the fine change. It is, therefore,necessary to determine a combination of the determination threshold Thand the quantization factor Q so that the image data after restorationhas a good image quality. Namely, the determination threshold Th isdetermined based on the quantization factor Q, and the quantizationfactor Q is determined based on the determination threshold Th.

The coding unit 143 performs a run-length conversion processing in eachblock for an AC coefficient after the quantization. The coding unit 143performs a differential processing between the blocks, and encodes theresultant data using entropy codes for the DC coefficient after thequantization. The entropy codes are a coding scheme having a code lengthaccording to an appearance probability, and Huffman codes are widelyknown as the entropy codes. The code table T2 holds a code table for theHuffman codes. The coding unit 143 performs the coding processing usingthis code table T2.

<Image Expander>

FIG. 9 is a block diagram of an example of a configuration of the imageexpander 21 shown in FIG. 3. FIG. 9 shows an example of detailedconfigurations of the JPEG decoder 23 and the image enlargementconverter 24.

[JPEG Decoder]

The JPEG decoder 23 includes a decoding unit 230, a dequantizer 231, aninverse DCT processor 232, a quantization table T1, and a code table T2.The JPEG decoder 23 performs inverse processings from those performed bythe JPEG encoder 14 to expand the compressed data, and restores thecompressed data to the image data before the JPEG compression. It isnoted that the quantization table T1 and the code table T2 need to bethe same data table as those included in the JPEG encoder 14. Thequantization table T1 and the code table T2 can be added to thecompressed data if it is necessary to do so, and transmitted from thetransmission side unit Ut to the reception side unit Ur.

The decoding unit 230 decodes Huffman codes in the compressed data usingthe code table T2, and decodes the DC coefficients subjected to thedifferential processing and the AC coefficients subjected to therun-length conversion. The decoded data is dequantized by thedequantizer 231 using the quantization table T1, thereby restoring thequantized DCT coefficient to the dequantized DCT coefficient for eachblock. This DCT coefficient is subjected to a inverse DCT processing bythe inverse DCT processor 232, thereby restoring the compressed imagedata to the image data before the JPEG compression processing, i.e., theimage data reduced by the image reduction converter 13.

[Image Enlargement Converter]

The image enlargement converter 24 includes a DS region divider 240, anarrangement restoring unit 241, and an interpolation processor 242 andperforms inverse processings from those performed by the image reductionconverter 13, thereby restoring the reduced data to the image databefore the reduction processing. First, the DS region divider 240divides the image data output from the JPEG decoder 23 into a pluralityof DS regions. As each of these DS regions, a DS region equal in size(16×16 pixels in the present embodiment) to each DS region used by theDS region divider 130 of the transmission side unit Ut is used.

The arrangement restoring unit 241 discriminates whether unreduced dataor reduced data is arranged in each divided DS region based on the fillregion present in the divided DS region, and based on thisdiscrimination result, the arrangement restoring unit 241 rearranges thereduced data and outputs the rearranged reduced data to theinterpolation processor 242.

FIGS. 10( a) to 10(e) show all arrangement states of the DS region ifthe DS region has the size of 16×16 pixels and the reduced data has thesize of 8×8 pixels. FIG. 10( a) shows an arrangement state if no fillregion is present in the DS region, and the arrangement restoring unit241 can discriminate that unreduced data is arranged in this DS regionfor the state shown in FIG. 10( a). The arrangement restoring unit 241can discriminate this state only by discriminating that a lower rightregion in the DS region is not the fill region. If FIGS. 10( b) to 10(d)show arrangement states one, two, three pieces of reduced data arearranged in the DS regions, respectively, FIG. 10( e) shows anarrangement state if the DS region consists only of fill regions. Thearrangement restoring unit 241 can discriminate each of these statesbased on the number of fill regions each at the size of 8×8 pixels.

If an arbitrary number of fill regions each at the size of 8×8 pixelsare arranged in the DS region at the size of 16×16 pixels, 2⁴arrangement patterns are considered. If the arrangement patterns shownin FIGS. 10( a) and 10(e) are subtracted from the 2⁴ arrangementpatterns, 14 arrangement patterns are present. Namely, four arrangementpatterns each including one fill region, six arrangement patterns eachincluding two fill regions, and four arrangement patterns each includingthree fill regions are present. Needless to say, therefore, arrangementpatterns are not limited to those shown in FIGS. 10( b) to 10(d) in thepresent embodiment but can be arbitrarily selected from the 14arrangement patterns.

The interpolation processor 242 interpolates pixels and increases thenumber of pixels for the reduced data output from the arrangementrestoring unit 241, thereby restoring the image data to the image dataat the original size before the downsampling (that is, the size of theDS region). As this interpolation processing, LP enlargement, linearinterpolation, quadratic interpolation, cubic interpolation or the likecan be performed. The LP enlargement is a method for restoring the imagedata to the image data before the downsampling by estimating the highfrequency component lost in the downsampling performed by thedownsampling unit. The interpolation processor 242 does not interpolatethe unreduced data. In this way, the image data is restored to the imagedata before the image reduction converter 13 performs the sizereduction, and the image data before the size reduction is output to theRGB converter 20.

The compressed image data in the YUV format consists of compressed dataof one luminance data Y and two pieces of compressed data on thecolor-difference data U and V, and the DS region divider 240, thearrangement restoring unit 241, and the interpolation processor 242perform their respective processings stated above on each of the threepieces of compressed data.

In the above-mentioned embodiment, the instance in which the image datais divided into a plurality of DS regions each at the size of 16×16pixels, and in which part of the DS regions are downsampled to those atthe size of 8×8 pixels has been described. However, the presentinvention is not limited to the instance of these numbers of pixels.

<32×32 DS Region>

FIGS. 11( a) to 11(c) show states of dividing the image data into aplurality of DS regions each at a size of 32×32 pixels, downsamplingpart of the DS regions to reduced data each at a size of 8×8 pixels, andrearranging the reduced data. FIG. 11( a) shows that the entire imagedata is divided into a plurality of DS regions each at the size of 32×32pixels. FIG. 11( b) shows that the DS regions that are not the importantregions are downsampled to reduce the number of pixels to ¼ bothvertically and horizontally, and that these DS regions are convertedinto reduced data each at the size of 8×8 pixels. FIG. 11( c) shows thatcontinuous pieces of reduced data are collected and rearranged in thesame DS region, thereby reducing the size of the entire image data.

In the example of FIGS. 11( a) to 11(c), the size of the entire imagedata after the reduction is larger than that in the example of FIGS. 7(a) to 7(c). However, if the entire image data is made far larger andmany continuous unimportant regions are present, the size of the entireimage regions can be made smaller than that according to the methodshown in FIGS. 7( a) to 7(c).

Second Embodiment

In the image compression and expansion system according to the firstembodiment, the arrangement restoring unit 241 of the image enlargementconverter 24 discriminates whether the unreduced data or the reduceddata is arranged in each DS region based on the position of the fillregion present in the DS region. Due to this, if a part of the unreduceddata or the reduced data coincides with the fill data, there is aprobability that the arrangement restoring unit 241 makes an erroneousdiscrimination. In the present embodiment, an image compression andexpansion system capable of preventing such erroneous discriminationwill be described.

Normally, a quantization error occurs to the image data compressed bythe JPEG encoder 14. Namely, if an arbitrary value is adopted as thefill data, the fill data inserted into the image data by the imagereduction converter 13 may possibly be changed to different data by thetime the image data is input to the image enlargement converter 24through the JPEG compression processing and the JPEG expansionprocessing. In this case, the arrangement restoring unit 241 can notaccurately discriminate the reduced data.

As possible first measures for avoiding this erroneous discrimination,the image enlargement converter 24 adopts a discrimination methodcapable of discriminating the fill data even if the quantization erroroccurs. Namely, if the arrangement restoring unit 241 discriminates thatpixel data in a range in which the pixel data has a width based on amaximum quantization error around the fill data is fill data, it ispossible to accurately discriminate the fill data even when thequantization error occurs to the image data.

As possible second measures, the image enlargement converter 24 adopts avalue free from the influence of the quantization error as the filldata. Namely, if one of quantization steps in the quantizer 142 isadopted as the fill data, then the fill data is not changed to adifferent value by the quantization, and the arrangement restoring unit241 can accurately discriminate the reduced data.

However, in the JPEG encoder 14, each quantization step width isobtained using the quantization table T1 and the quantization factor Q,and each quantization step is determined based on this quantization stepwidth. Due to this, if the quantization table T1 and the quantizationfactor Q are changed, then the maximum quantization error and thequantization step are changed, and the fill data should be changed.Nevertheless, only a value zero that serves as a reference value basedon which the quantization step is obtained from the quantization stepwidth always remains equal to the quantization step despite thequantization step width. It is, therefore, preferable to adopt thesecond measures and to adopt the value 0 during the quantization as thefill data.

In the JPEG encoder 14, a central value is subtracted from each pixeldata to shift a level of the pixel data, the level-shifted pixel data isconverted into decoded data, and the resultant decoded data is subjectedto the DCT processing for convenience of an arithmetic processing. Inthe JPEG decoder 23, the central value is added to each pixel data thathas been subjected to the inverse DCT processing, thereby restoring thepixel data to the original pixel data. Therefore, this central value maybe adopted as the fill data.

Normally, each pixel data is expressed by eight bits and a value of thepixel data ranges from zero to 255, due to this, in the JPEG encoder 14,the central value 128 is subtracted from each pixel data and the pixeldata is converted into the signed data. Namely, a DC coefficient of zeroat the time of the quantization corresponds to the average value 128 ofthe pixel data. Therefore, if the fill data is the central value 128 ofthe pixel data, then all the DCT coefficients during the quantizationare all zero, and the fill data is free from the influence of thequantization error despite the quantization step width.

As already stated, the compression and expansion processings accordingto the JPEG standards (“JPEG compression and expansion processings”)employ the quantization table T1 (see FIG. 8) having the largerquantization step width at the higher frequency component. For thisreason, the image data after the restoration through the JPEGcompression and expansion processings may possibly include blocksnarrowed down only to a DC component during the compression. In such ablock, all the pixels are coincident with one another in value after therestoration and the coincident value is an average value (where thevalue is an integer) of the pixel data before the compression.Accordingly, if the average value of the pixel data that constitutes theJPEG block is a value closer to the fill data, all the pixels in theJPEG block coincide with the fill data after the pixel data is subjectedto the JPEG compression and expansion processings, as a result, it isoften impossible to discriminate which is the fill region. Namely, thearrangement restoring unit 241 erroneously discriminates the fillregion, with the result that the image enlargement converter 24 maypossibly be unable to accurately restore the compressed image data tothe original image data.

To prevent such fill region erroneous discrimination, it is necessarythat the rearrangement unit 132 processes the JPEG block for which theaverage value of the pixel data is closer to 128 so that the averagevalue is farther from 128. Specifically, a maximum change width of theaverage value in the block by the JPEG compression and expansionprocessings is smaller than α, the pixel data is changed so that theaverage value is either 128+α or 128−α for the block for which thedifference between the average value before the JPEG compression and 128is smaller than α. For instance, if the pixel data in the block is f(x,y) and satisfies the following expression 5, it is necessary to changethe average value of the image data to 128−α in advance and if the pixeldata f(x, y) satisfies the following expression 6, it is necessary tochange the average value of the image data to 128+α in advance.

$\begin{matrix}{{128 - \alpha} < {\frac{1}{64}{\sum\limits_{x = 0}^{7}{\sum\limits_{y = 0}^{7}{f( {x,y} )}}}} \leq 128} & (5) \\{128 \leq {\frac{1}{64}{\sum\limits_{x = 0}^{7}{\sum\limits_{y = 0}^{7}{f( {x,y} )}}}} < {128 + \alpha}} & (6)\end{matrix}$

A method for determining such a α will be described. In the DCTprocessing according to the JPEG, the DC coefficient is calculated asrepresented by the following equation 7. Due to this, if the averagevalue in the JPEG block is changed only by α, the DC coefficient ischanged only by 16α. For instance, if α is added to each pixel data, theDC coefficient is increased only by 16α.

$\begin{matrix}{{DC} = {\frac{1}{4}{\sum\limits_{x = 0}^{7}{\sum\limits_{y = 0}^{7}{f( {x,y} )}}}}} & (7)\end{matrix}$

As stated above, the quantizer 142 quantizes the DC coefficient by thequantization step width determined according to the quantization tableT1 and the quantization factor Q. For the DC coefficient, the data onthe upper left end of the quantization table T1 shown in FIG. 8 is used.This data is normally 16 and 16Q is used as the quantization step width.In this case, a quantization error of the DC coefficient except for anerror related to operation accuracy is smaller than 16Q. Accordingly, aslong as 16α≧16Q (that is α≧Q) is satisfied, the average value in theblock after the JPEG compression and expansion processings is notchanged to 128 if the average value in the block is 128−α or 128+α.

Considering the influence of the change of the pixel data on the imagequality, it is preferable that α is as small as possible. From theseviewpoints, it is preferable that α is Q (α=Q). However, at α<2Q, theimage data after the restoration is basically equal to the image data atα=Q, therefore, in view of the operation accuracy error, it is morepreferable to determine α within a range of Q<α<2Q.

Specifically, the rearrangement unit 132 applies four JPEG blocks (eachat the size of 8×8 pixels) that constitute the unreduced data at thesize of 16×16 pixels and the reduced data at the size of 8×8 pixels toeach of the expressions 5 and 6. As a result, if the expression 5 issatisfied, the same value is added to each pixel in the block so thatthe average value is equal to 128+α. If the expression 6 is satisfied,the same value is subtracted from each pixel in the block so that theaverage value is equal to 128−α.

Third Embodiment

In the first embodiment, the instance of dividing the entire image intothe DS regions at the equal size has been described. In the presentembodiment, by contrast, an instance of dividing the entire image intoDS regions at different sizes and using different reduction rates indownsampling will be described.

FIGS. 12( a) to 12(c) show an example of an operation performed by animage compression and expansion system according to the third embodimentof the present invention.

FIG. 12( a) shows a state in which a DS region divider 130 divides theimage into DS regions. The DS region divider 130 divides the entireimage into DS regions with a mixture of two or more different sizes. Inthe present embodiment, large DS regions each at a size of 32×32 pixelsand small DS regions each at a size of 16×16 pixels are mixed together.On this occasion, the DS region divider 130 divides the image so thatthe important region is included in small DS regions. In addition, theDS region divider 130 divides the image so that a part of theunimportant region away from the important region is included in largeDS regions and a part thereof closer to the important region is includedin small DS regions.

Specifically, the DS region divider 130 divides image data with a sizeof 96×96 pixels input from the image input apparatus 101 and into sevenlarge DS regions (#1 to #3, #4, #7, #10, and #13) and eight smallregions (#5, #6, #8, #9, #11, #12, #14, and #15). In addition, the DSregion divider 130 outputs image data of the respective DS regions tothe downsampling unit 131. Among the image regions, the DS regions #9,#12, and #15 are the important regions, each of which is divided intosmall DS regions.

FIG. 12( b) shows a state in which the downsampling unit 131 performsdownsampling. The downsampling unit 131 performs downsampling on eachunimportant DS region to convert the image data of the DS region intoreduced data at a size of 8×8 pixels. Namely, the downsampling unit 131performs the downsampling on each DS region at a reduction rateaccording to the size of the DS region, and converts the image data ofeach DS region into the reduced data at the equal size despite sizes ofthe respective DS regions.

Specifically, for the large DS regions (#1 to #3, #4, #7, #10, and #13)each at the size of 32×32 pixels obtained by dividing the image by theDS region divider 130, the downsampling unit 131 performs downsamplingthereon to reduce the number of pixels to ¼ both vertically andhorizontally. For the small DS regions (#5, #6, #8, #11, and #14) eachat the size of 16×16 pixels obtained by dividing the image by the DSregion divider 130, the downsampling unit 131 performs downsamplingthereon to reduce the number of pixels to ½ both vertically andhorizontally. As a result, a plurality of pieces of unreduced data (#9,#12, and #15) each at the size of 16×16 pixels and a plurality of piecesof reduced data each at the size of 8×8 pixels are output to therearrangement unit 132.

FIG. 12( c) shows a state in which the rearrangement unit 132 rearrangesdata. The rearrangement unit 132 rearranges the reduced data in eachsmall DS region (at the size of 16×16 pixels). The downsampling unit 131discriminates the reduced data downsampled to ¼ in size (¼ reduced data)from the reduced data downsampled to ½ in size (½ reduced data).Therefore, the rearrangement unit 132 rearranges the data so as not tomix up the ¼ reduced data and the ½ reduced data in the same DS region.

Further, arrangement of the fill region and the reduced data in each DSregion is made different between an instance of arranging the ¼ reduceddata and an instance of arranging the ½ reduced data so that the ½reduced data and the ¼ reduced data can be discriminated from each otherbased on the fill region(s). Specifically, if the ½ reduced data is tobe arranged in the DS region, then the fill region is arranged at alower right position of the DS region, and the pieces of reduced dataare arranged at an upper left position, an upper right position, and alower left position in this order, similarly to the first embodiment. Ifthe ¼ reduced data is to be arranged in the DS region, then the fillregion is arranged at an upper left position of the DS region, and thepieces of reduced data are arranged at the lower right position, thelower left position, and the upper right position in this order.

Since the pieces of reduced data #1 to #3 are ¼ reduced data, the fillregion is arranged at the upper left position of the DS region at a sizeof 16×16 pixels, and the respective pieces of reduced data are arrangedfrom the lower right position in a predetermined order. Since thereduced data #4 is also ¼ reduced data, it is similarly arranged in thenext DS region. Since subsequent pieces of reduced data #5 and #6 are ½reduced data, they are not arranged in the same DS region as that inwhich the reduced data #4 is arranged but arranged in a new DS region.Accordingly, the fill data is inserted into each blank region of the DSregion in which the reduced data #4 is arranged, a fill region isarranged at a lower right position in the next DS region, and the piecesof reduced data #5 and #6 are arranged in the next DS region from theupper left position in a predetermined order. Since the next reduceddata #7 is ¼ reduced data, it is arranged in a different DS region fromthat in which the pieces of reduced data #5 and #6 are arranged.Thereafter, pieces of unreduced data and reduced data are sequentially,similarly arranged in the DS regions.

If FIG. 12( c) is compared with FIG. 11( c), it is understood that theentire image can be further reduced although the important regions arenot subjected to downsampling and each unimportant region is subjectedto downsampling to be converted into the ¼ reduced data.

FIGS. 13( a) to 13(h) show all arrangement states of the DS region ifthe DS region has the size of 16×16 pixels and the reduced data has thesize of 8×8 pixels. FIG. 13( a) shows an arrangement state if no fillregion is present in the DS region, and it can be discriminated that theunreduced data is arranged. This discrimination can be performed only bydiscriminating that an upper left region and a lower right region in theDS region are not the fill regions. FIGS. 12( b) to 12(d) showarrangement states if three, two, and one pieces of ½ reduced data arearranged in the DS regions, respectively, and FIGS. 12( e) to 12(g) showarrangement states if three, two, and one pieces of ¼ reduced data arearranged in the DS regions, respectively, and FIG. 12( h) shows a statein which the DS region consists only of fill regions. The arrangementrestoring unit 241 can discriminate each of these states based on thearrangement of the fill regions each at the size of 8×8 pixels.Arrangement methods for the instances of FIGS. 12( b) to 12(g) are notlimited to this embodiment but can be arbitrarily selected from the 14arrangement patterns stated above.

Fourth Embodiment

In the third embodiment, the instance of dividing the entire image intothe DS regions at different sizes and using different reduction ratesamong the DS regions at the different sizes in the downsampling has beendescribed. In the present embodiment, by contrast, an instance ofdividing the entire image into DS regions at an equal size andperforming downsampling hierarchically will be described.

In the present embodiment, the downsampling unit 131 performsdownsampling twice (or three or more times) on a part of the DS regionsthat do not contain important regions. Namely, for a plurality of DSregions, the downsampling unit 131 performs downsampling thereon toobtain a plurality of pieces of reduced data thereafter, thedownsampling unit 131 performs downsampling again on a group of thesepieces of reduced data. In this case, for each DS region that has beensubjected to downsampling twice, the same result can be obtained as thatif the DS region is subjected to downsampling at a reduction rate twiceas high as that for the former downsampling except for the operationaccuracy-related error and a difference resulting from the filterprocessing. In other words, even if hierarchical downsampling isperformed, substantially the same result as that according to the thirdembodiment can be obtained.

FIGS. 14( a) to 14(c), 15(d) and 15(e) show an example of an operationperformed by an image compression and expansion system according to thefourth embodiment of the present invention. FIG. 14( a) shows a state inwhich the DS region divider 130 divides the image into DS regions. TheDS region divider 130 divides the entire image into DS regions (#1 to#36) each at a size of 16×16 pixels exactly equally to the firstembodiment. Among the DS regions, the DS regions #22, #28, and #34 arethe important regions.

FIG. 14( b) shows a state in which the downsampling unit 131 performsfirst downsampling. The downsampling unit 131 performs downsampling oneach of the DS regions, which is the unimportant region, to convert theimage data of each of the DS regions into ½ reduced data at a size of16×16 pixels. The operation stated so far is the same as that accordingto the first embodiment.

FIG. 14( c) shows a state of grouping the pieces of reduced data. Thedownsampling unit 131 groups four adjacent ½ reduced data, therebygenerating a new DS region at a size of 16×16 pixels. Namely, thedownsampling unit 131 collects adjacent reduced data #1, #2, #7, and #8to generate a new DS region #A, and collects adjacent reduced data #3,#4, #9, and #10 to generate a new DS region #B. Likewise, thedownsampling unit 131 generates new DS regions #A to #G. Duringgrouping, the downsampling unit 131 leaves the reduced data #15, #16,#21, #27, and #C33 that can not be grouped, uncollected.

FIG. 15( d) shows a state in which the downsampling unit 131 performssecond downsampling. The second downsampling is performed only on thegrouped new DS regions, and ¼ reduced data at a size of 8×8 pixels isgenerated for each of these regions. At the time the second sampling iscompleted, the state is equal to the state shown in FIG. 12( b)according to the third embodiment.

FIG. 15( e) shows a state in which the rearrangement unit 132 performsdata rearrangement. The rearrangement unit 132 discriminates the ¼reduced data downsampled twice from the ½ reduced data downsampled once,and rearranges the reduced data so as not to arrange a mixture of the ¼reduced data and the ½ reduced data in the same DS region. Thearrangement methods are the same as those according to the thirdembodiment.

Fifth Embodiment

In the above mentioned embodiment, the image reduction converter 13 andthe image enlargement converter 24 perform their respective processingssimilarly on each of the luminance data Y and the color-difference dataU and V. The processings on these respective pieces of data have beendescribed without discrimination. In the present embodiment, an instancein which a processing on the luminance data Y is made different fromprocessings on the color-difference data U and Y will be described.

FIG. 16 is a block diagram of an example of a configuration of animportant element of an image compression apparatus according to thefifth embodiment of the present invention and shows a detailedconfiguration of the image reduction converter 13 shown in FIG. 2. Thisimage reduction converter 13 includes a DS region divider 130 common tothe data Y, U, and V, three downsampling units 131 and 131 scorresponding to the respective data Y, U, and V, and threerearrangement units 132 and 132 s corresponding to the respective dataY, U, and V.

The DS region divider 130 divides the image into DS regions for each ofthe luminance data Y and the color-difference data U and V and performsthe same DS region dividing processing as that according to the firstembodiment. Namely, for each of the data Y, U, and V, the DS regiondivider 130 divides the image into DS regions so that the same pixeldata belongs to the same DS region.

The downsampling unit 131 is an adaptive downsampling unit that performsdownsampling on the luminance data Y. The downsampling unit 131determines whether the target DS region is the important region or not,and performs downsampling based on this determination result, similarlyto the first embodiment (see FIG. 6). On the other hand, the twodownsampling units 131 s perform downsampling on the color-differencedata U and V, respectively. Each of the downsampling units 131 sdetermines whether to perform downsampling according to thedetermination result of the downsampling unit 131 for the luminancedata.

FIG. 17 is a block diagram of an example of a detailed configuration ofthe downsampling unit 131 s shown in FIG. 16. This downsampling unit 131s differs from the adaptive downsampling unit 131 shown in FIG. 6 inthat the downsampling unit 131 s does not include the subtracter 33 forcontrolling the output selector 32 and the downsampling determinationunit (“DS determination unit”) 34. Due to this, the output selector 32operates based on a determination result of the DS determination unit 34in the downsampling unit 131 for the luminance data Y. Namely, ifdownsampling is performed on the same DS region for the luminance dataY, downsampling is also performed thereon for the color-difference dataU and V, and if downsampling is not performed on the DS region for theluminance data Y, downsampling is not performed on the color-differencedata U and V, either.

The rearrangement unit 132 rearranges the reduced data for the luminancedata Y. Similarly to the first embodiment, the rearrangement unit 132rearranges the reduced data and arranges a discrimination regionconsisting of the fill data at a predetermined position in the DS regionin which the reduced data is arranged. The rearrangement unit 132 srearranges the reduced data for each of the color-difference data U andV. During this rearrangement, even if the reduced data is arranged inthe DS region, the rearrangement unit 132 s does not rearrange thediscrimination region consisting of the fill data in the DS region.Therefore, as compared with the luminance data Y, more pieces of reduceddata can be arranged in the same DS region.

FIGS. 18( a) and 18(b) show an example of an operation performed by theimage compression and expansion system according to the fifth embodimentof the present invention. FIGS. 18( a) and 18(b) relate to the sameimage data as that shown in FIG. 7 (according to the first embodiment),FIG. 18( a) shows a state of rearranging the luminance data Y, and FIG.18( b) shows a state of rearranging the color-difference data U and V.

Since the rearrangement processing related to the luminance data Y isperformed by the same method as that described in the first embodiment,it will not be described herein. For the color-difference data U and V,at the time of rearranging continuous pieces of reduced data in the DSregions, four pieces of reduced data are arranged in one DS region. Dueto this, if the downsampling unit 131 s inputs five or more pieces ofreduced data continuously to the corresponding rearrangement unit 132 s,the rearrangement unit 132 s arranges fifth and the following pieces ofreduced data in the next DS region. Namely, until the number of piecesof reduced data reaches a maximum number of pieces of reduced data thatcan be arranged in one DS region, the pieces of reduced data are packedin each DS region. On the other hand, if the number of pieces ofcontinuous reduced data is three or less, the fill data is inserted intoeach blank region. In FIG. 18( b), the pieces of reduced data #1 to #4are arranged in a first DS region, and the pieces of reduced data #5 to#8 are arranged in a next DS region. Thereafter, continuous pieces ofreduced data #1 to #21 are similarly arranged in the DS regions four byfour. The other processings are the same as those for the luminance dataY.

In this case, for the rearranged color-difference data U and V, nodiscrimination regions each consisting of the fill data are arranged. Itis, therefore, impossible to discriminate whether the data is reduced orunreduced based on the rearranged color-difference data U and V duringrearrangement and restoration. According to the present embodiment, bycontrast, the DS region to be subjected to downsampling is common to thecolor-difference data U and V and the luminance data Y. In addition, theJPEG compressed color-difference data U and V are transmitted togetherwith the JPEG compressed luminance data Y from the transmission sideunit Ut to the reception side unit Ur. It is, therefore, possible todiscriminate whether the data is reduced or unreduced for the rearrangedcolor-difference data U and V based on the rearranged luminance data Y,and restore arrangement to original arrangement of the color-differencedata U and V.

FIG. 19 is a block diagram of an example of a configuration of animportant element of the image expansion apparatus according to thefifth embodiment of the present invention and shows a detailedconfiguration of the image enlargement converter 24 shown in FIG. 3.This image enlargement converter 24 includes the DS region divider 240common to the data Y, U, and V, three arrangement restoring units 241and 241 s corresponding to the respective data Y, U, and V, and aninterpolation processor 242 common to the data Y, U, and V.

The DS region divider 240 divides each of the luminance data Y and thecolor-difference data U and V output from the JPEG decoder 23 into aplurality of DS regions and performs the same DS region dividingprocessing as that according to the first embodiment.

The arrangement restoring unit 241 performs restoration of thearrangement of the luminance data Y, and similarly to the firstembodiment, discriminates whether a target DS region is reduced data orunreduced data. Based on this discrimination result, the arrangementrestoring unit 241 restores the arrangement of the data to the originalarrangement. The two arrangement restoring units 241 s performrestoration of the arrangement of the color-difference data U and V,respectively. Each arrangement restoring unit 241 s discriminateswhether a target DS region is reduced data or unreduced data based onthe discrimination result of the arrangement restoring unit 241 for theluminance data.

Namely, the arrangement of the reduced data and the unreduced data isdetermined uniquely if an appearance order of these pieces of data isdetermined. Due to this, it is possible to recognize the arrangement ofthe reduced data and the unreduced data for the rearrangedcolor-difference data U and V from the discrimination result of thearrangement restoring unit 241 for the luminance data Y. The arrangementrestoring units 241 s thus perform restoration of the arrangement of thecolor-difference data U and V, respectively.

Sixth Embodiment

In the fifth embodiment, the instance in which the YUV converter 10converts the image data in the RGB format into the image data in theYUV444 format, and in which the number of pieces of data for theluminance data after conversion is equal to the number of pieces of datafor the color-difference data after conversion has been described. Onthe contrast, in the present embodiment, an instance in which the YUVconverter 10 converts the image data in the RGB format into the imagedata in the YUV422, YUV420 format or the like, and in which the numberof pieces of data for the luminance data after conversion is differentfrom the number of pieces of data for the color-difference data afterconversion will be described.

Normally, a human eye is lower in discrimination power for a colorcomponent than for a luminance component, and lower in discriminationpower for the horizontal direction than for the vertical direction. Insuch a format as the YUV422 or YUV420 format, the number of pieces ofdata for the color-difference data is set smaller than that of pieces ofdata for the luminance data using these visual characteristics. Namely,for the image data in the YUV422, YUV420 format or the like convertedfrom the RGB format by the YUV converter 10, the number of pieces ofdata for the luminance data Y does not coincide with the number ofpieces of data for the color-difference data U and V. Accordingly, ifthe DS region divider 130 divides the luminance data Y and thecolor-difference data U and V into the DS regions at the equal size,respectively, the DS regions for the color-difference data U and V donot coincide with the respective DS regions for the luminance data Y. Inthe following description, each DS region for the color-difference dataU and V will be referred to as a “color-difference DS region” and eachDS region for the luminance data Y will be referred to as a “luminanceDS region” so as to discriminate them from each other if it is necessaryto do so.

FIGS. 20( a) and 20(b) show an example of the image data in the YUV422format. In the YUV422 format, the number of pieces of horizontal datafor the color-difference data U and V is half as large as that of piecesof horizontal data for the luminance data Y. Each color-difference dataU and V is made to correspond to two horizontally adjacent pieces ofluminance data Y. Due to this, each color-difference DS regioncorresponds to horizontally adjacent two luminance DS regions.

FIG. 20( a) shows a DS region dividing state for the luminance data Y,and FIG. 20( b) shows a DS region dividing state for the luminance dataY. Image data in the YUV422 format at a size of 96×96 pixels isconstituted by the luminance data Y at a size of 96×96 pixels and thecolor-difference data U and V each at a size of 48×96 pixels. The DSregion divider 130 divides each of the luminance data Y and thecolor-difference data U and V into a plurality of DS regions each at asize of 16×16 pixels. Thus, the luminance data Y is divided into 36luminance DS regions (#1 to #36), each of the color-difference data Uand V is divided into 18 color-difference DS regions (#1 to #18). Inthis case, a color-difference DS region #n corresponds to luminance DSregions #2n−1 and #2n (where n is an integer from 1 to 18).

Each of the downsampling units 131 s for the color-difference data U andV determines whether or not to perform downsampling on thecolor-difference DS region #n based on determination results of thedownsampling unit 131 related to the two luminance DS regions #2n−1 and#2n corresponding to the color-difference DS region #n. Specifically, ifboth of the two luminance DS regions #2n−1 and #2n are the importantregions, the downsampling unit 131 s determines that thecolor-difference DS region #n is the important region and does notperform downsampling thereon. On the other hand, if one of or both ofthe two luminance DS regions #2n−1 and #2n are the unimportant regions,the downsampling unit 131 s determines that the color-difference DSregion #n is the unimportant region and performs downsampling thereon.In FIG. 20, since the luminance DS regions #21, #22, #27, #28, #33, and#34 are the important regions, the color-difference regions #11, #14,and #17 are the important regions. However, as to whether to performdownsampling on the color-difference DS region #n if one of the twoluminance DS regions #2n−1 and #2n is the important region and the otheris the unimportant region can be arbitrarily set in advance.

FIGS. 21( a) and 21(b) show an example of an operation performed by theimage compression and expansion system according to the sixth embodimentof the present invention. FIG. 21( a) shows a data rearranging state forthe luminance data Y shown in FIG. 20. FIG. 21( b) shows a datarearranging state for the color-difference data U and V shown in FIG.20.

The rearranging unit 132 rearranges the reduced data output from thedownsampling unit 131 so that the number of pieces of horizontal datafor the luminance data Y input to the image reduction converter 13coincides with that for the luminance data Y output from the imagereduction converter 13. In FIG. 21( a), the rearrangement unit 132rearranges the reduced data so that the number of pieces of horizontaldata is 96.

Exactly in the same manner as that for the rearrangement unit 132, therearrangement unit 132 s rearranges the reduced data output from thedownsampling unit 131 s so that the number of pieces of horizontal datafor the color-difference data U or V input to the image reductionconverter 13 coincides with that for the color-difference data U or Voutput from the image reduction converter 13. In FIG. 21( b), therearrangement unit 132 s rearranges the reduced data so that the numberof pieces of horizontal data is 48.

In the present embodiment, the color-difference DS region does notcoincide with the luminance DS region. However, according to the YUVformat, the color-difference DS region is made to correspond to theluminance DS region. Due to this, similarly to the fifth embodiment, itis possible to discriminate whether the data is reduced data orunreduced data for the rearranged color-difference data U and V based ondiscrimination information included in the rearranged luminance data Y,and restore arrangement of the data to original arrangement.

Some JPEG compressor (e.g., JPEG chip) include the YUV converter 10 thatconverts the RGB format into the YUV422 format in front of the JPEGencoder 14, and adopt the RGB format as the format of input image data.If such a JPEG compressor is employed as the image compressor 11 shownin FIG. 2, the image data in the YUV format output from the imagereduction converter 13 is temporarily converted into the image data inthe RGB format, and the resultant image data is input to the JPEGcompressor.

In this case, the YUV converter 10 provided in front of the imagereduction converter can select an arbitrary format such as the YUV444,YUV422 format and the like. If the YUV422 format is selected, the imagereduction converter 13 operates similarly to that according to theabove-mentioned embodiment. If the YUV444 format is adopted, it ispreferable to set a size of the reduced data in the downsamplingprocessing in view of re-conversion of the YUV444 format into the YUV422format in the JPEG compressor. That is, it is preferable that for thecolor-difference data, two horizontally adjacent DS regions are combinedinto a new DS region, and that this new DS region is subjected todownsampling.

It is noted, however, that if the DS region divider 130 sets a size ofeach of the DS regions related to the luminance data Y and thecolor-difference data U and V, and the image is divided into the new DSregions mentioned above for the color-difference data U and V, it isunnecessary to perform the combining processing. For instance, if theYUV422 format is adopted, then the DS region divider 130 divides theimage into a plurality of DS regions each at a size of 16×16 pixels forthe luminance data Y and divides the image into a plurality of DSregions each at a size of 32×16 pixels, which is twice as large as thenumber of horizontal pixels of the DS region for the luminance data Y,for the color-difference data.

Seventh Embodiment

In the fifth and the sixth embodiments, the instance in which pieces ofreduced data of the color-difference data U and V are packed andrearranged in one color-difference DS region, and in which pieces ofunreduced data thereof are not rearranged has been described. Accordingto the present embodiment, by contrast, an instance of rearranging boththe reduced data and the unreduced data will be described.

FIG. 22 is a block diagram of an example of a configuration of an imagereduction converter 13 according to the seventh embodiment of thepresent invention. This image reduction converter 13 differs from thatshown in FIG. 16 (according to the fifth embodiment) in that theconverter 13 includes two unreduced data dividers 134.

Each of the unreduced data dividers 134 divides unreduced data outputfrom the downsampling unit 131 s for the color-difference data U or Vinto a plurality of pieces of data equal in size to those for thereduced data, and outputs the divided unreduced data to therearrangement unit 132 s. The rearrangement unit 132 s rearranges thereduced data and the unreduced data without discriminating them fromeach other exactly in the same manner as that for the reduced dataaccording to the fifth or sixth embodiment. Namely, pieces of reduceddata or unreduced data are packed and rearranged in eachcolor-difference DS region until the number of pieces of data reaches amaximum number of pieces of data that can arranged in onecolor-difference DS region.

FIGS. 23 and 24 show an example of an operation performed by an imagecompression and expansion system according to the seventh embodiment ofthe present invention. FIGS. 23( a) to 23(c) show an example of theoperation up to an unreduced data dividing processing performed in theimage reduction converter 13 shown in FIG. 22. FIG. 23( a) shows anexample of the color-difference data U or V output from the DS regiondivider 130. This color-difference data U or V is the same as that shownin FIG. 20 (according to the sixth embodiment). The DS region divider130 divides the color-difference data U or V into a plurality ofcolor-difference DS regions #1 to #18 each at a size of 16×16 pixels.Among these, the color-difference DS regions #11, #14, and #17 areassumed as the important regions and the other color-difference DSregions are assumed as the unimportant regions.

FIG. 23( b) shows reduced data and unreduced data output from thedownsampling unit 131 s. The downsampling unit 131 s performsdownsampling on each unimportant region to convert the image data in theunimportant region into reduced data at a size of 8×8 pixels. On theother hand, the downsampling unit 131 s does not perform downsampling oneach important region and outputs pieces of data each at a size of 16×16pixels as the unreduced data.

FIG. 23( c) shows reduced data and divided data output from eachunreduced data divider 134. The unreduced data divider 134 outputs thereduced data input from the corresponding downsampling unit 131 swithout processing it. In addition, the unreduced data divider 134divides the unreduced data into four pieces of reduced data each equalin size (8×8 pixels) to the reduced data. In FIG. 23( c), each of theunreduced data #11, #14, and #17 that are not subjected to downsamplingis divided into four pieces of unreduced data to generate 12 pieces ofdivided data #11 a to #11 d, #14 a to #14 d, and #17 a to #17 d.

FIGS. 24( a) and 24(b) show an example of an operation performed by therearrangement unit 132 s shown in FIG. 22. This rearrangement unit 132 srearranges the reduced data and the divided data. During thisrearrangement, the rearrangement unit 132 s can rearrange four pieces ofreduced data or divided data in each color-difference DS regionsimilarly to the fifth embodiment, however, in the present embodiment,the rearrangement unit 132 s sequentially arranges the pieces of reduceddata or divided data in the horizontal direction. Namely, therearrangement unit 132 s starts arrangement at an upper left part of theimage data, and sequentially arranges pieces of reduced data or divideddata from left to right in the horizontal direction. When reaching aright end of the image data, the rearrangement unit 132 s shiftsarrangement position to a downward direction, and sequentially arrangespieces of reduced data or divided data from left to right in thehorizontal direction again. The rearrangement unit 132 s thus repeatedlyperforms this operation. If pieces of divided data are to be arranged,the rearrangement unit 132 s continuously arranges four pieces ofdivided data that constitute one unreduced data.

If the image reduction converter 13 thus rearranges the color-differencedata U or V in the horizontal direction, it is unnecessary to divide thedata into a plurality of color-difference DS regions when the imageenlargement converter 24 restores the rearranged color-difference data Uand V. Due to this, the color-difference data U and V output from theJPEG decoder 23 is input to the arrangement restoring unit 241 s withoutvia the DS region divider 240.

The unreduced data that is not subjected to downsampling is larger insize than the reduced data. However, if the unreduced data is dividedinto a plurality of pieces of divided data each equal in size to thereduced data, even if a mixture of the reduced data and the unreduceddata is present, it is possible to pack these pieces of data withoutblanks and rearrange them. Accordingly, it suffices that the fill datais added to a lowermost portion of the image data so as to reshape theimage data into rectangular data, so that the size of thecolor-difference data U and V after the rearrangement can be madesmaller.

During restoration in the image enlargement converter 24, thearrangement restoring unit 241 performs not only restoration of thearrangement of the reduced data but also that of the arrangement of thedivided data. Needless to say, a divided data arrangement restoringprocessing can be performed based on the discrimination informationincluded in the luminance data Y similarly to the reduced dataarrangement restoring processing.

FIGS. 25 and 26 are explanatory views for comparing the image reductionconverter according to the sixth embodiment with that according to theseventh embodiment. FIG. 25 shows an example of image data in which theimportant regions and the unimportant regions repeatedly appear in thehorizontal direction. This is image data in the YUV422 format at a sizeof 128×96 pixels. Each of the luminance DS region and thecolor-difference DS region has a data size of 16×16 pixels. Theluminance data Y is divided into 48 luminance DS regions #1 to #48, andeach of the color-difference data U and V is divided into 24color-difference DS regions #1 to #24. Since the important regions andthe unimportant regions repeatedly appear in the horizontal direction ofthis image data, a conspicuous difference is generated between theprocessing result of the image reduction converter 13 according to thesixth and seventh embodiments.

FIGS. 26( a) and 26(b) show output data if the image data shown in FIG.25 is input to the image reduction converter 13 according to the sixthand seventh embodiments. FIG. 26( a) shows a state after therearrangement according to the sixth embodiment, and FIG. 26( b) shows astate after the rearrangement according to the seventh embodiment.

As already stated, in the image reduction converter 13 according to thesixth embodiment, it is necessary to insert the discrimination regioninto the luminance data Y but unnecessary to insert the discriminationregion into each of the color-difference data U and V. Due to this, ifthe size of the luminance data Y is compared with that of each of thecolor-difference data U and V after the rearrangement, thecolor-difference data U and V are normally smaller in size than theluminance data Y. However, in the image data shown in FIG. 25, theimportant regions and the unimportant regions repeatedly appear in thehorizontal direction, therefore, if the number of pieces of horizontaldata for the luminance data Y is different from that for thecolor-difference data U or V similarly to the image data in the YUV422format, the above-mentioned size relationship after the rearrangementbetween the luminance data Y and the color-difference data U or V afterthe rearrangement is often inverted.

In FIG. 26( a), the luminance data Y is reduced while thecolor-difference data U or V is not reduced. To cause the JPEG encoder14 to perform the compression processing, it is necessary to make avertical length of the luminance data Y coincident with that of thecolor-difference data U or V. Therefore, the rearrangement unit 132 alsoadds the fill data to an end of the luminance data Y (a part surroundedby a broken line). As a result, the image data is not at all reduced bythe image reduction converter 13.

In FIG. 26( b), by contrast, the luminance data Y and thecolor-difference data U and V are both reduced, and the image reductionconverter 13 reduces the image data. In the seventh embodiment, theunreduced data for the color-difference data U or V is divided into aplurality of pieces of divided data, and the pieces of divided data arerearranged together with the pieces of reduced data efficiently. Due tothis, even if the important regions and the unimportant regionsrepeatedly appear, inversion of the compression rate does not occurdifferently from the sixth embodiment.

Each of the image compressor 11 and the image expander 21 according toeach of the embodiments can be provided as a computer program executableon a calculator such as a personal computer. Further, each of the JPEGencoder 14 and the JPEG decoder 23 can be constituted by general-purposehardware such a JPEG chip or a PC add-on board that can performprocessing at high rate. Each of the image reduction converter 13 andthe image enlargement converter 24 can be constituted by a computerprogram. These computer programs are provided by storing them in anoptical storage medium such as a CD-ROM, a magnetic storage medium suchas a flexible disc, or a semiconductor storage medium such as an ICmemory. Furthermore, these computer programs can be provided through anelectric communication line such as the Internet or the LAN (Local AreaNetwork).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example of a configuration of an imagecompression and expansion system according to a first embodiment of thepresent invention.

FIG. 2 is a block diagram of an example of a configuration of atransmission side unit Ut shown in FIG. 1.

FIG. 3 is a block diagram of an example of a configuration of areception side unit Ur shown in FIG. 1.

FIG. 4 is a block diagram of an example of a configuration of an imagecompressor 11 shown in FIG. 2, showing an example of detailedconfigurations of an image reduction converter 13 and a JPEG encoder 14.

FIGS. 5( a) and 5(b) show examples of a DS region division and adownsampling performed by the image compressor 11.

FIG. 6 is a block diagram of an example of a detailed configuration of adownsampling unit 131.

FIGS. 7( a) to 7(c) show an example of an operation performed by theimage reduction converter 13.

FIG. 8 shows examples of a quantization table T1.

FIG. 9 is a block diagram of an example of a configuration of an imageexpander 21 shown in FIG. 3, showing an example of detailedconfigurations of a JPEG decoder 23 and an image enlargement converter24.

FIGS. 10( a) to 10(e) show all arrangement states in a DS region if theDS region has a size of 16×16 pixels and reduced data has a size of 8×8pixels.

FIGS. 11( a) to 11(c) show states of dividing image data into aplurality of DS regions each at a size of 32×32 pixels, downsamplingpart of the DS regions to reduced data each at a size of 8×8 pixels, andrearranging the reduced data.

FIGS. 12( a) to 12(c) show an example of an operation performed by animage compression and expansion system according to a third embodimentof the present invention.

FIGS. 13( a) to 13(h) show all arrangement states in a DS region if theDS region has a size of 16×16 pixels and reduced data has a size of 8×8pixels.

FIGS. 14( a) to 14(c) and 15(d) to 15(e) show an example of an operationperformed by an image compression and expansion system according to afourth embodiment of the present invention.

FIGS. 15( d) and 15(e) show an example of the operation performed by theimage compression and expansion system subsequent to FIGS. 14( a) to14(c).

FIG. 16 shows a detailed configuration of an image reduction converter13 according to a fifth embodiment of the present invention.

FIG. 17 is a block diagram of an example of a detailed configuration ofa downsampling unit 131 s shown in FIG. 16.

FIGS. 18( a) and 18(b) show an example of an operation performed by animage compression and expansion system according to the fifth embodimentof the present invention.

FIG. 19 shows a detailed configuration of an image enlargement converter24 according to the fifth embodiment of the present invention.

FIGS. 20( a) and 20(b) show an example of the image data in a YUV422format.

FIGS. 21( a) and 21(b) show an example of an operation performed by animage compression and expansion system according to a sixth embodimentof the present invention.

FIG. 22 is a block diagram of an example of a configuration of an imagereduction converter 13 according to a seventh embodiment of the presentinvention.

FIGS. 23( a) to 23(c) show an example of an operation up to an unreduceddata dividing processing performed in the image reduction converter 13shown in FIG. 22.

FIGS. 24( a) and 24(b) show an example of an operation performed by arearrangement unit 132 s shown in FIG. 22.

FIG. 25 shows an example of image data in which important regions andunimportant regions repeatedly appear in a horizontal direction.

FIGS. 26( a) and 26(b) show output data if the image data shown in FIG.25 is input to the image reduction converter 13 according to the sixthembodiment and that according to the seventh embodiment, respectively.

DESCRIPTION OF REFERENCE NUMERALS

-   13 Image reduction converter-   14 JPEG encoder-   23 JPEG decoder-   24 Image enlargement converter-   130 DS region divider-   131, 131 s Downsampling unit-   132, 132 s Rearrangement unit-   134 Unreduced data divider-   140 Block divider-   141 DCT processor-   142 Quantizer-   143 Coding unit-   T1 Quantization table-   T2 Code table-   230 Decoding unit-   231 Dequantizer-   232 Inverse DCT processor-   240 DS region divider-   241, 241 s Arrangement restoring unit-   242 Interpolation processor

1. An image compression apparatus comprising: downsampling regiondividing means for dividing image data into downsampling regions each ata size that is an integer multiple of a block size; downsampling meansfor performing downsampling on at least a part of said downsamplingregions to decrease the number of pixels thereof, and for generatingreduced data each at a size that is an integer multiple of said blocksize; rearranging means for arranging two or more pieces of said reduceddata in each of said downsampling regions subjected to saiddownsampling, for inserting fill data into each of pixels left in saiddownsampling region in which said reduced data is arranged, and forremoving said downsampling regions in which said reduced data is notarranged; and data compressing means for dividing image data generatedby said rearranging means into block regions each at said block size,and for performing a compression processing including an orthogonaltransform and a quantization on each of said block regions, therebygenerating compressed image data, wherein said downsampling regiondividing means divides said image data into said downsampling regionshaving different sizes, and said downsampling means performs saiddownsampling at different compression rates according to said sizes ofsaid downsampling regions, thereby generates said reduced data equal insize.
 2. An image compression apparatus comprising: downsampling regiondividing means for dividing image data into downsampling regions each ata size that is an integer multiple of a block size; downsampling meansfor performing downsampling on at least a part of said downsamplingregions to generate reduced data, for collecting said reduced data togenerate a new downsampling region, and for performing said downsamplingagain on said new downsampling region, thereby generating new reduceddata each at a size that is an integer multiple of said block size;rearranging means for arranging two or more pieces of said reduced datain each of said downsampling regions subjected to said downsampling, forinserting fill data into each of pixels left in said downsampling regionin which said reduced data is arranged, and for removing saiddownsampling regions in which said reduced data is not arranged; anddata compressing means for dividing image data generated by saidrearranging means into block regions each at said block size, and forperforming a compression processing including an orthogonal transformand a quantization on each of said block regions, thereby generatingcompressed image data.
 3. An image compression apparatus comprising:downsampling region dividing means for dividing luminance data thatconstitute image data into luminance downsampling regions each at a sizethat is an integer multiple of a block size, and for dividingcolor-difference data that constitute said image data intocolor-difference downsampling regions each at a size that is the integermultiple of said block size; luminance downsampling means for performingdownsampling on at least a part of said luminance downsampling regionsto decrease the number of said luminance data, and for generatingreduced luminance data each at a size that is an integer multiple ofsaid block size; luminance data rearranging means for arranging two ormore pieces of said reduced luminance data in each of said luminancedownsampling regions subjected to said luminance downsampling, and forinserting fill data into each of pixels left in said luminancedownsampling region in which said reduced luminance data is arranged;color-difference downsampling means for performing downsampling on saidcolor-difference downsampling regions determined based on said luminancedownsampling regions subjected to said luminance downsampling todecrease the number of said color-difference data, and for generatingreduced color-difference data each at a size that is the integermultiple of said block size; color-difference data rearranging means forarranging two or more pieces of said reduced color-difference data ineach of said color-difference downsampling regions; data compressingmeans for dividing said luminance data and said color-difference datagenerated by said luminance data rearranging means and saidcolor-difference data rearranging means, respectively, into blockregions each at said block size, and for performing a compressionprocessing including an orthogonal transform and a quantization on eachof said block regions, thereby generating compressed image data, whereinsaid luminance data rearranging means arranges said reduced luminancedata in each of said luminance downsampling regions while leaving adiscrimination region having said block size at a predetermined positionin each of said luminance downsampling regions.
 4. The image compressionapparatus according to claim 3, further comprising: unreduced datadividing means for dividing each of said color-difference downsamplingregions that are not subjected to the downsampling into regions each ata size equal to said reduced color-difference data, thereby generatingdivided color-difference data, wherein said color-difference datarearranging means rearranges said reduced color-difference data and saiddivided color-difference data.